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Description 

BACKGROUND OF THE INVENTION 
1. Field of the invention 
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The present invention relates to a picture coder. 
and^eUcuW^ 

Hy degraded by a data error or a data loss caused ou ^ 
ing picture data transmission. 

2 Description of the prior art 

and video-on-demand (VOD). nu i 
w 963 and MPEG (Moving Picture Experts Group) are 
we^rSJi asSnationa. standardization standards^ 
S Se. ihe coding method adopted by the rTU-T 
Recommendation H.263. shown '^£2 

Multimedia Coding" by Hirosh, Yasuda. pp. 84 97. 

which perlorms the inter-frame codjngof^ch frame by 
referencing the immediately preceding *~ 
that all the frames be transmitted n_ *e co^ 
sequence. For a telephone line or an ISDN line ov 
Sh data is transmitted after a connection with i part- 
nertesSolished. data reachesthe partner w.thouttoss 

whicSTaS i s divided into smaii units (called pa*ets o 
~S before it is transmitted, there is a possibility that 

In general, networks employ a protocol <™ 
Die TCP • Transmission Control Protocol) in which the 
Knotting device sends packets with attach* sena. 
numbers, and the receiving device rearranges the pacK 
etet the correct sequence, confirms their arrival and 
sends requests for the retransmission of ron-a^v.ng 
Z£ IZSS* the transmitting device ln«JJ.« 
XTese problems and increase network rel,^. 

However, when network operation is unstable and 
pac^e* dropped frequent., « ^edeC 
tHs type of protocol can cause large cumulat rve aeays 
to taMu* which is inappropriate for the real-Ume trans- 
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unintended data with an incorrect data number. This 
may lead to an unexpected operation. 

To solve the problem of negative acknowledgment 
signals, it is possible to estimate the correct numbers 
from correct data received after the error and to send 
the acknowledgment signals at a time. However, this 
method prevents acknowledgment signals from being 
sent until the end of incorrect data numbers is detected, 
causing a delay and preventing quick recovery from 
degraded picture quality. 

Thus, to successfully transmit moving-picture data 
over a network in which a data dropout occurs fre- 
quently, there has been a long felt need for a picture 
coding system which ensures high coding efficiency 
and quick recovery from degraded picture quality. 

SUMMARY OF THE INVENTION 

To solve the above problems, a picture coder 
according to the present invention includes coding 
means for coding input pictures through predictive cod- 
ing, one at a time, based on a reference picture and for 
outputting coded data, transmission means for transmit- 
ting the coded data coded by the coding means and a 
frame number thereof to a picture decoder, and refer- 
ence picture updating means for controlling updating of 
the reference picture based on a decoding status signal 
sent from the picture decoder and an associated frame 
number. The picture coder comprises: 

That is, the picture coder comprises means which 
uses an acknowledgment signal (ACK : Acknowledg- 
ment signal) as the decoding status signal and which 
returns the acknowledgment signal when there is no 
transmission or decoding error. The means determines 
that a decoding error has occurred if the acknowledge 
signal is not received within a specified time. And, as a 
next reference picture, the reference picture updating 
means selects one of pictures transmitted correctly 
before the decoding error occurred. 

The picture coder according to the present inven- 
tion uses a decoder which comprises reception means 
for receiving coded data coded thorough predictive cod- 
ing and a frame number (or frame number and block 
number) of a reference picture of the coded data over a 
transmission line, decoding means for decoding coded 
data, received by the reception means, with the use of a 
frame number (or frame number and block number) of 
the reference picture, and decoding status signal trans- 
mission means for outputting to the picture coder a 
decoding status signal representing a status of decoded 
data decoded by the decoding means and a frame 
number (or frame number and block number) corre- 
sponding to the data. 

Thus, the picture decoder sends decoding status 
information on received coded data to the picture coder 
and, during coding, uses one of correctly-decoded 
frames (or blocks) as the reference picture. Therefore, 
the picture decoder can decode a P frame without hav- 



ing to wait for the next I frame. 

In addition, the acknowledgment signal is used as 
the decoding status signal. This enables a correct frame 
number (or frame number and block number) to be 
5 returned to the coder, allowing the coder to perform 
processing correctly. At the same time, elimination of 
the need to wait for data after an error minimizes a 
delay. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The present invention will be understood more fully 
from the detailed description and the accompanying 
drawings below of the preferred embodiments of the 
is invention. 

In the drawings: 



FIG. 1 is a block diagram showing an example of a 
functional configuration of a picture coder (sending 
device) used in a first embodiment. 
FIG. 2 is a block diagram showing an example of 
detailed functional configuration of a reference 
frame updating module used in the first embodi- 
ment 

FIG. 3 is a conceptual diagram showing conven- 
tional coding. 

FIG. 4 is a conceptual diagram showing an example 
of coding when a transmission error occurs in a 
conventional system. 

FIG. 5 is a conceptual diagram showing an example 
of coding in the first embodiment 
FIG. 6 is a conceptual diagram showing an example 
of coding when an ACK signal transmission error 
occurs in the first embodiment. 
FIG. 7 is a block diagram showing an example of 
functional configuration of a picture coder (sending 
device) used in a second embodiment. 
FIG. 8 is a block diagram showing an example of 
detailed functional configuration of a reference 
frame updating module used in the second embod- 
iment. 

FIG. 9 is a conceptual diagram showing an example 
of coding when a transmission error occurs in the 
second embodiment. 

FIG. 10 is a conceptual diagram showing an exam- 
ple of coding when a transmission error occurs in 
an active mode. 
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DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

(A) First embodiment 

Referring to the attached drawings, there is shown 
a configuration, operation, and effect of a moving-pic- 
ture coder (sending device) used in the first embodi- 
ment 

The moving-picture coder according to the present 
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invention updates a reference picture on a frame basis 
or on a block basis, where a block is one partition of a 
frame. When picture data is processed on a frame 
basis, a decoding status signal is returned and the ref- 
erence picture is updated each time a frame is updated; s 
when picture data is processed on a block basis, the 
decoding status signal is returned and the reference 
picture is updated each time a block is updated. The 
processing and configuration described below apply to 
frames and blocks, although only frames are mentioned w 
in some part of the description 

The picture coder (sending device) in the first 
embodiment is characterized in that new means is pro- 
vided to check the status of the acknowledgment (ACK) 
signal. That is, the means determines that no error has is 
occurred on a frame or a block for which the acknowl- 
edgment signal is received and that a decoding error 
has occurred on a frame or a block for which the 
acknowledgment signal is not received. The picture 
coder updates reference frames according to the ACK 20 
signal only. 

A functional configuration of the moving-picture 
coder (sending device) used in the first embodiment is 
shown in FIG. 1 . 

As shown in FIG. 1, a moving-picture coder 100 25 
comprises a moving-picture input module 101. a coding 
module 102, a decoding module 103 . a frame memory 
module 104, a reference frame memory module 105. a 
intra/inter determination module 106. a coded data 
transmitting module 107. an acknowledgment signal 30 
receiving module 108, a decoding status determination 
module 109, and a reference frame updating module 
110. 

The moving-picture input module 101 passes mov- 
ing-picture data from an input source, such as a video 35 
camera, to the coding module 102, one frame at a time. 

The coding module 102 codes entered frame data, 
one frame at a time or one block at a time, and passes 
coded data to the decoding module 103 and the coded 
data transmitting module 107. When coding data, the 40 
coding module 102 switches processing between intra- 
frame coding and inter-frame coding according to an 
instruction from the intra/inter determination module 
106 . where Intra" refers to intra-frame coding and 
"inter refers to inter-frame coding. When the coding 4s 
module 102 performs inter-frame coding, it references 
frame data stored in the reference frame memory mod- 
ule 105. 

The decoding module 103 decodes coded data and 
writes decoded data, as well as frame numbers, into the so 
frame memory module 104. 

The frame memory module 104 is a memory 
medium, such as a memory circuit, which memorizes 
data for each frame. Each time a reference frame is 
updated by the reference frame updating module 110, 55 
unnecessary data is cleared from the frame memory 
module 104.. 

The reference frame memory module 105, a mem- 



ory medium such as a memory circuit which contains 
reference frame data for use in inter-frame coding, is 
updated by the reference frame updating module 1 10. 

The intra/inter determination module 106 selects 
which coding to use: intra-frame coding or inter-frame 
coding. Normally, intra-frame coding is selected at reg- 
ular intervals (e.g., once every thirty frames) and, at 
other times, inter-frame coding is selected; when the 
intra/inter determination module 106 receives a refresh 
signal from a receiving device (moving-picture decoder 
not shown in the drawing), it forces intra-frame coding 
and passes information to that effect to the coding mod- 
ule 102 and the reference frame updating module 110 

The coded data transmitting module 107 sends 
coded moving-picture data to one or more receiving 
devices (moving-picture decoders). The coded data 
transmitting module 107 multiplexes control information, 
such as an intra/inter flag or a reference frame number 
onto the data; it also multiplexes sending device infor- 
mation and receiving device information, as necessary, 
onto the data. 

The acknowledgment signal receiving module 108 
receives the acknowledgment signal (decoding status 
signal), which indicates that which frames (or blocks) 
have been decoded, from the receiving device (moving- 
picture decoder), and passes the received decoding 
status signal to the reference frame updating module 
110. 

The decoding status determination module 109 
determines that a frame (or a block) for which the 
acknowledgment signal (ACK signal) is received within 
a specified time has no decoding error and that a frame 
(or a block) for which the acknowledgment signal is not 
received within a specified time has a decoding error. It 
then passes the acknowledgment status and the frame 
number (or frame number and block number) to the ref- 
erence frame updating module 1 10. 

The reference frame updating module 1 10 updates 
the reference frame, responsive to the instructions 
received from the intra/inter determination module 106 
and the decoding status determination module 1 09. The 
reference frame updating module 110 usually updates 
the reference frame each time it receives a frame; when 
a decoding error occurred, the module returns the refer- 
ence frame to a most recent frame (or block) for which 
acknowledgment signal was received. Upon receiving 
an intra-frame coding instruction from the intra/inter 
determination module 106, the reference frame updat- 
ing module 110 copies the data of the current frame 
from the frame memory module 104 to the reference 
frame memory module 105 and deletes ail the frame 
data from the frame memory module 104. 

The reference frame updating module 1 1 0 is further 
divided into detailed functional blocks. FIG. 2 is a 
detailed functional configuration diagram of the refer- 
ence frame updating module 110. As shown in FIG. 2, 
the reference frame updating module 1 10 comprises a 
frame number setting module 111, a last-received- 
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frame number detecting module 112, a candidate 
number deleting module 1 13. a reference frame candi- 
date number recording module 114, and a data copy 
module 115. 

When the frame number setting module 111 
receives information indicating that a decoding error 
has occurred from the decoding status determination 
module 109, it passes the incorrect-frame number (or 
frame number and block number) to the last-received- 
frame number detecting module 1 12 and the candidate 
number deleting module 113. Upon receiving a last- 
received-frame number from the last-received-frame 
number detecting module 112, the frame number set- 
ting module 111 sets up the last-received-frame number 
as the number of frame (or the number of frame and the 
number of block) to be copied, and passes the number 
to the data copy module 115. 

When the frame number setting module 111 
receives information indicating that There is no candi- 
date number, * it determines that no error occurred and 
suspends processing. 

When the frame number setting module 111 
receives information indicating that no decoding error 
has occurred from the decoding status determination 
module 109, it passes the correct-frame number (or 
frame number and block number) to the candidate 
number deleting module 113. 

The last-received-frame number detecting module 
112 searches a candidate number list stored in the ref- 
erence frame candidate number recording module 114 
to check to see if an incorrect-frame number received 
from the frame number setting module 1 11 is recorded 
in the list 

When the incorrect-frame number received from 
the frame number setting module 1 1 1 is recorded in the 
candidate frame number list, the last-received-frame 
number detecting module 112 finds a last-received- 
frame number and returns it to the frame number setting 
module 111. On the other hand, when the incorrect- 
frame number received from the frame number setting 
module 111 is not recorded in the candidate frame 
number list the last-received-frame number detecting 
module 112 returns information indicating that "there is 
no candidate number" to the frame number setting mod- 
ule 111. 

When the candidate number deleting module 113 
receives an incorrect^frame number (or frame number 
and block number) from the frame number setting mod- 
ule 1 1 1 , it deletes the incorrect-frame number (or frame 
number and block number) and the frame numbers 
added later than the frame number from the candidate 
frame number list stored in the reference frame candi- 
date number recording module 114. This keeps only 
correct-frame numbers in the reference frame candidate 
number recording module 114. At this time, it is also 
possible to delete frame data corresponding to the 
frame number from the frame memory module 104. 

When the candidate number deleting module 113 



receives a correct-frame number (or frame number and 
block number) from the frame number setting module 
1 1 1 , it records the number in the reference frame candi- 
date number recording module 114 as a last-received- 

5 frame number. It also deletes the number (or frame 
number and block number) and the frame numbers ear- 
lier than the frame number from the candidate frame 
number list recorded in the reference frame candidate 
number recording module 114. At this time, it is also 

w possible to delete frame data corresponding to the 
deleted frame number from the frame memory module 
104. 

The reference frame candidate number recording 
module 114 manages the numbers of frame data stored 

15 in the frame memory module 104. It records a last- 
received-frame number, which is the number of a frame 
to be used as a reference frame, and a list of candidate 
frame numbers. A list may be created for each block. 
Thus, each time latest frame data coded by the coding 

20 module 102 is recorded in the frame memory module 
104, the reference frame candidate number recording 
module 114 adds the frame number to the candidate 
list. The candidate frame number list, which is in the ref- 
erence frame candidate number recording module 114 

25 in the above description, may be recorded in the frame 
memory module 104 as header information. 

The data copy module 115 usually copies latest 
frame data from the frame memory module 104 to the 
reference frame memory module 105. Upon receiving a 

30 last-received-frame number from the frame number set- 
ting module 111. the data copy module 115 copies the 
corresponding frame data from the frame memory mod- 
ule 104 to the reference frame memory module 105. 
Next, the operation of the picture coder (sending 

35 device) used in the first embodiment is described. 

The moving-picture coder checks the decoding sta- 
tus signal, which is returned from a moving-picture 
decoder, to switch a reference picture used in inter- 
frame predictive coding. The moving-picture coder uses 

40 the acknowledgment signal (ACK signal) as the decod- 
ing status signal. When a decoding error occurs, this 
signal is not returned. 

The moving -picture coder updates the reference 
picture each time it receives a frame while no decoding 

as error occurs. This is the same coding method as that 
proposed by H.263 in which the immediately-preceding 
frame is always referenced during coding. When a 
decoding error is detected, the last frame received cor- 
rectly is used as the reference picture. 

so FIG. 5 shows an example of operation that is per- 
formed by the moving-picture coder which has the refer- 
ence frame updating module built-in. For simplicity, 
each frame is composed of one block. For a multiple- 
block frame, the coder does the same operation for 

55 each block. 

While no decoding error occurs, the moving-picture 
coder updates the reference frame each time it receives 
a frame. When the ACK signal arrives, the coder 
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records the number of the frame (or block) as the last- 
received-frame number. For example, when frame e is 
coded, the last-received-frame number is frame c. 
When a transmission error occurs on frame d, the mov- 
ing-picture decoder cannot decode frame d and there- 
fore does not return the ACK signal for frame d. When 
the ACK signal is not received, the moving-picture coder 
detects that a decoding error has occurred on frame d 
and changes the reference frame to the frame corre- 
sponding to last-received-frame number (frame c) and 
codes frame f. 

By doing so, the moving-picture decoder can 
decode the next frame f and the following frames, 
although frame e cannot be decoded because it is 
already coded and transmitted before the decoding 
error is detected. This allows the system to quickly 
recover from a decoding error caused by a transmission 
error. 

When a decoding error is detected on frame d, the 
moving-picture coder deletes the numbers of frame d 
and all the later frames (only frame e in the example in 
the figure) from the reference frame candidate number 
list. This prevents frame d which is incorrect and frame 
e which references frame d and therefore causes a 
decoding error from being selected as a reference pic- 
ture. In addition, as explained below, decoding error 
processing for frame e may be omitted. At this time, it is 
possible to delete frame data corresponding to the 
deleted numbers from the frame memory module 104. 
This enables a small-memory system to delete data 
which becomes unnecessary, eliminating the possibility 
of clearing necessary data due to a shortage of memory 
and thereby ensuring coding efficiency. 

The ACK signal is not returned for frame e because 
it cannot be decoded by the moving-picture decoder. 
Therefore, the moving-picture coder detects a decoding 
error on frame e. If. at this time, the reference frame is 
changed to a previous frame (c) for frame e, frame g 
also references frame c, with degraded coding effi- 
ciency. However, because frame e is already deleted 
from the reference frame candidate list, information indi- 
cating "there is no candidate number" is returned. 
Therefore, the reference frame is not returned to a 
frame with the last-received-frame number, with no deg- 
radation in coding efficiency. 

FIG. 6 shows an example of coding when an ACK 
signal error occurs. When an ACK transmission error 
occurs on frame d, the moving-picture coder detects 
that a decoding error has occurred on frame d, changes 
the reference frame to a frame (c) with the last-received- 
frame number, and then codes frame f. Although the 
moving-picture decoder has received frame d correctly, 
the reference frame is changed to a previous frame and 
therefore the coding efficiency of frame f is degraded to 
some degree. However, it should be noted that all the 
frame are decoded and little or no effect is found on 
other frames. In a system where the negative acknowl- 
edgment signal (NACK signal) is used as the decoding 



status signal, a NACK signal error generates a frame 
that cannot be decoded, causing somewhat serious 
effects. On the other hand, in a system where the 
absence of the ACK signal indicates a decoding error, 

5 an ACK signal transmission error would minimize cod- 
ing efficiency degradation to such a level that almost 
every person cannot notice. 

As described above, when no transmission error 
occurs in a system of the first embodiment, the refer- 

10 ence frame is updated each time a frame is received. 
Thus, the coding efficiency is the same as that of other 
coding methods such as that proposed by ITU-T Rec- 
ommendation H.263. 

When a transmission error occurs, decoding correct - 
is ness of a frame already coded cannot be guaranteed 

but the subsequent frames can be decoded before the 

next I frame arrives and therefore coding efficiency is 

not degraded. 

In addition, use of the acknowledgment signal (ACK 
20 signal) as the decoding status error makes it possible to 

transmit frame numbers and block numbers correctly 

and to eliminate the need to wait for correct data after 

an error, thus minimizing the delay. 

Even if an ACK signal transmission error occurs, coding 
25 efficiency is somewhat degraded but all the frames are 

decoded. 

Another advantage is that, when continuous decod- 
ing errors are detected, the reference frame candidate 
list used in this embodiment bypasses the processing of 
30 changing the reference frame to a previous frame when 
the second error occurs, ensuing coding efficiency. 

Variations of the first embodiment are: 

(1) In the above embodiment, the reference frame 
35 updating module 110 copies frame data from the 

frame memory module 104 to the reference frame 
memory module 105 when intra-frame coding is 
performed. This need not always be done. For 
example, the intra/inter determination module 106 

40 may send an intra-frame coding request signal to 
the decoding module 103 and, at the same time, 
the decoding module 103 may write frame data 
directly into the reference frame memory module 
105 during intra-frame coding. This enables copy 

45 processing to be bypassed and makes the configu- 
ration simple. 

(2) In FIG. 1, the reference frame memory module 
105 and the frame memory module 104 are sepa- 
rate modules. This configuration is not always 

so required. Frame data may be stored in a memory 
module, with the reference frame being pointed to 
by a pointer. This configuration eliminates the need 
for copying data from the frame memory module 
104 to the reference frame memory module 105 

55 during intra-frame coding; simply moving the 
pointer points to the reference frame, thereby mak- 
ing the circuit configuration simple. 
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(B) Second embodiment 

Referring to the attached drawings, there is shown 
a configuration and operation of a moving-picture coder 
(sending device) used in the second embodiment. 

The following describes a reference frame update 
method which uses an acknowledgment signal (ACK 
signal) but which is different from a method used in the 
first embodiment. A moving-picture coder used in the 
second embodiment uses only the ACK signal and 
switches the reference frame update method between 
the method different from the first embodiment and the 
method used in the first embodiment according to the 
status of a transmission line. 

FIG. 10 shows an example of operation of the 
update method different from that used in the first 
embodiment. This method is characterized in that a ref- 
erence picture is updated only when the ACK signal is 
received but not when the ACK signal is not received. 

For example, for each of frames (a, b, c). the ACK 
signal is sent from the receiving device to indicate that it 
has been received correctly. Each time the ACK signal 
is received, the reference frame is switched and the next 
frame is coded. 

When a error occurs during transmission of frame 
d, the ACK signal is not sent and therefore frame d is not 
used as a reference frame. Instead, frame c. which was 
already received correctly, is used as a reference frame 
and the next frame f is coded. 

This update method always performs coding with a 
correctly-received reference frame as the reference 
frame. Therefore, the receiving device can decode a P 
frame without having to watt for an I frame that will be 
sent later, with little or no degradation in coding effi- 
ciency even when a frame is lost. However, if the round 
trip delay in the period of time from when the sending 
device sends coded data to the time the ACK signal is 
received is large, the ACK signal for the previous frame 
is not received before the coding of the next frame is 
started. This means that, even when there is no trans- 
mission error, a frame that is two or more frames before 
the current frame is referenced and so the coding effi- 
ciency is more or less degraded. 

The update method shown in FIG. 10 updates the 
reference frame when no transmission error occurs, 
while the update method shown in the first embodiment 
updates the reference frame when a transmission error 
occurs. In the following description, the former (the 
update method shown in FIG. 10) is referred to as an 
Active mode, while the latter (the update method shown 
in the first embodiment) as a Negative mode. The Active 
mode is effective when the transmission status is good; 
the Negative mode is effective when the transmission 
status is bad. Thus, switching these two modes accord- 
ing to the transmission status ensures high coding effi- 
ciency. 

The moving-picture coder (sending device) used in 
the second embodiment comprises a transmission sta- 



tus determination module 203 which determines the 
status of the transmission line with the use of the decod- 
ing status signal, an update method switching module 
204 which switches the update mode between the 

5 Active mode and the Negative mode according to the 
transmission line status, and a reference frame adaptive 
updating module 202 which updates the reference 
frame adaptively 

When the transmission line status is good, the Neg- 

10 ative mode is enabled to update the reference frame 
each time a frame is received; in this case, only when a 
transmission error occurs, the reference frame is 
returned to the last reference frame that was positively 
acknowledged. On the other hand, when the transmis- 

15 sion line status is bad, the Active mode is enabled; in 
this case, when a frame is acknowledged, the reference 
frame is updated. 

FIG. 7 is a functional configuration diagram of a 
moving-picture coder (sending device) in the second 

20 embodiment Note that, for a module in FIG. 7 with a 
function equivalent.to that of a module shown in FIG. 1, 
the corresponding number is used and no description is 
given here. The following describes only new modules: 
decoding status determination module 201, reference 

25 frame adaptive updating module 202, transmission sta- 
tus determination module 203, and update method 
switching module 204. 

The decoding status determination module 201, 
like the decoding status determination module 109, 

30 determines that a frame (or a block) for which the 
acknowledgment signal (ACK signal) is received within 
a specified time has no decoding error and that a frame 
(or a block) for which the acknowledgment signal is not 
received within a specified time has a decoding error. 

35 based on the decoding status signal from the acknowl- 
edgment signal receiving module 108. It then passes 
the acknowledgment status and the frame number (or 
block number) to the reference frame updating module 
110 and the transmission status determination module 

40 203. 

The reference frame adaptive updating module 202 
updates the reference frame according to instructions 
from the intra/inter determination module 106, decoding 
status determination module 201, and update method 
45 switching module 204. 

There are two reference frame update methods: 
Active mode and Negative mode. In the Active mode, a 
frame (or block) on which no decoding error occurred is 
used as the reference frame picture; in the Negative 
so mode, the reference frame picture is updated each time 
a frame (or block) is received and, when a decoding 
error occurs, the reference frame is returned to the last 
frame on which no decoding error occurred. 

The reference frame adaptive updating module 202 
55 switches the mode between the Active mode and the 
Negative mode according to an instruction from the 
update method switching module 204. 

For example, upon receiving information indicating 
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"there is no decoding error" from the decoding status 
determination module 201 in the Active mode, the refer- 
ence frame adaptive updating module 202 copies frame 
data (or block data) from the frame memory module 104 
to the reference frame memory module 105. At this 5 
time, data corresponding to numbers earlier than the 
frame number (or frame number and block number) may 
be deleted from the frame memory module 104. When 
information indicating that "there is a decoding error" is 
received, the no operation is performed. At this time, the 10 
frame number (or block number) may be deleted from 
the reference frame candidate number list, and data 
corresponding to the number may be deleted from the 
frame memory module 104. 

In the Negative mode, the reference frame adaptive 15 
updating module 202 usually copies frame data (or 
block data) from the frame memory module 104 to the 
reference frame memory module 105 each time frame 
data (or block data) is received. The module does this 
operation in synchronization with the coding module 20 
102. When information indicating that "there is no 
decoding error" is received from the decoding status 
determination module 201, the last-received-frame 
number (or frame number and block number) which is 
the number of the last frame positively acknowledged is 25 
updated. At this time, data earlier than the last-received- 
frame number may be deleted from the frame memory 
module 104. When information indicating "there is a 
decoding error" is received, frame data (or block data) 
corresponding to the last-received-frame number is 30 
copied from the frame memory module 104 to the refer- 
ence frame memory module 105 in order to change the 
reference frame to the previous frame with that number. 

Upon receiving an intra-frame coding request from 
the intra/inter determination module 106, the reference 35 
frame adaptive updating module 202 copies current 
frame data from the frame memory module 1 04 to the 
reference frame memory module 105 and deletes all 
frame data from the frame memory module 104. 

The transmission status determination module 203 40 
determines the current transmission status based on 
the decoding status information received from the 
decoding status determination module 201 . and then 
passes the result to the update method switching mod- 
ule 204. The transmission status is either "good" or 45 
"bad." For example, if a transmission error was detected 
in M frames (or M blocks) out of the last N frames (or N 
blocks), the status is "bad"; otherwise (N s M), the sta- 
tus is "good." 

In addition, if a transmission error occurs in consec- so 
utive L frames (or L blocks), the status is "bad." There 
may be a plurality of determination criteria at the same 
time. In that case, the status is "good" if all the determi- 
nation results are good, and is "bad" if one of determi- 
nation result is "bad." The status may be determined to 55 
be "bad" only if all the results are "bad." 

The update method switching module 204 switches 
the update method to be used by the reference frame 



adaptive updating module 202, based on the transmis- 
sion status determined by the transmission status deter- 
mination module 203. For example, when the 
determination result is "good," the module switches the 
mode to the Negative mode; when the determination 
result is "bad." the module switches the mode to the 
Active mode. 

The following describes the function of the refer- 
ence frame adaptive updating module 202 with refer- 
ence to a detailed functional block diagram. 

Fig. 8 is an example of detailed functional configu- 
ration of the reference frame adaptive updating module 
202. Note that, for a module in FIG. 8 with a function 
equivalent to that of a module shown in the detailed 
configuration of the reference frame updating module 
1 10 in the first embodiment, the corresponding number 
is used and no description is given here. The following 
describes only a frame number adaptive setting module 
205 which is a new module. 

The operation of the frame number adaptive setting 
module 205 depends on the update mode that is 
switched by the update method switching module 204. 

For example, in the Active mode, when the frame 
number adaptive setting module 205 receives informa- 
tion indicating that there is no decoding error from the 
decoding status determination module 201 , it sets up a 
frame number (or frame number and block number), 
which is sent with that information, as the number of a 
frame to be copied and passes the number to the data 
copy module 1 15. At the same time, the frame number 
adaptive setting module 205 passes the frame number 
(or frame number and block number) of the positively 
acknowledged frame to the candidate number deleting 
module 113. 

When the frame number adaptive setting module 
205 receives from the decoding status determination 
module 201 a determination result indicating that a 
decoding error has occurred, it passes the number of 
incorrect frame (or frame number and block number) to 
the candidate number deleting module 113. 

In the Negative mode, when the frame number 
adaptive setting module 205 receives from the decoding 
status determination module 201 a determination result 
indicating that a decoding error has occurred, it passes 
the number of the incorrect frame (or frame number and 
block number) to the last-received-frame number 
detecting module 112 and the candidate number delet- 
ing module 113. When the frame number adaptive set- 
ting module 205 receives a last-received-frame number 
from the last-received-frame number detecting module 
1 12, it sets up the received last-received-frame number 
as the number of frame to be copied as the reference 
picture and then passes the number to the data copy 
module 115. 

When the frame number adaptive setting module 
205 receives information indicating that "there is no can- 
didate number" from the last-received-frame number 
detecting module 112, it determines that no decoding 
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error occurred and suspends processing. 

When the frame number adaptive setting module 
205 receives from the decoding status determination 
module 109 a determination result indicating that there 
is no decoding error, it passes the number of correct 
frame (or frame number and block number) to the can- 
didate number deleting module 113. 

The moving-picture coder (sending device) used in 
the second embodiment is described below. 

In the second embodiment, the sending device 
checks the transmission status through the use of an 
acknowledgment signal sent from a receiving device. 
Inter-frame coding is performed using one of reference 
frame update modes that is selected according to the 
transmission status. 

Operation of a receiving device does not depends 
on the mode. That is, the receiving device simply sends 
an acknowledgment signal (ACK signal) to the sending 
device. As described in the first embodiment, the receiv- 
ing device updates a reference frame based on a refer- 
ence frame number and a block number multiplexed on 
coded data, and repeats decoding. 

The sending device checks the status of the trans- 
mission line through the use of the ACK signal and, 
depending upon the status, switches the update mode. 

Normally, the sending device updates the reference 
frame picture in the Negative mode and, based on the 
updated reference frame, performs picture coding. 

The transmission status determination module 203 
continually monitors the transmission status using one 
or more criteria. Upon detecting that the status is "bad." 
the mode is switched the mode from the Negative mode 
to the Active mode that is provided for use when the 
transmission status is bad. In the example in FIG. 9, the 
transmission status is determined to be "bad" after two 
consecutive transmission errors. 

That is, when a transmission error occurs on frame 
d, frame e cannot be decoded because frame d could 
not be decoded successfully. In this case, the ACK sig- 
nal cannot be received two consecutive times and 
therefore the transmission status is determined to be 
"bad." Then, the mode is changed to the Active mode 
when the next frame, g, is coded while the reference 
picture, which is not yet updated, is still frame c. 

As described above, when the reference frame 
update mode is changed to the Active mode, the refer- 
ence frame is updated when the ACK signal is received. 
When the ACK signal is not received, the reference 
frame is not updated. 

When the transmission status determination mod- 
ule 203 determines in the Active mode that the trans- 
mission status is "good", the mode returns to the 
Negative mode. In FIG. 9, the transmission status is 
determined to be "good" when four consecutive correct 
frames are received. 

That is, when the ACK signal is received for four 
consecutive frames (I, m, n, and o), the transmission 
status is determined to be "good." At this point, the 



mode changes to the Negative mode and coding is per- 
formed while referencing the immediately-preceding 
frame p. 

Thus, while there is no decoding error, the refer- 

s ence frame picture is updated in the Negative mode 
each time a frame is received. This does not degrade 
coding efficiency even when the transmission delay is 
targe. Another problem with the Negative mode is that a 
frame immediately following an incorrect frame cannot 

10 be decoded. However, as long as a decoding error 
occur less frequently, degradation of picture quality is 
not so large; in fact, because the picture quality during a 
non-error period is dominant in the Negative mode, pic- 
ture quality degradation is not large. 

is On the other hand, in the Active mode which is 
used when a decoding error occurs frequently, all 
frames are decoded with no large degradation in picture 
quality. In particular, when a decoding error occurs 
repeatedly, coding in the Active mode gives a result 1 - 

20 1 .5 dB better than the result of the conventional coding 
method or the Negative mode. 

In the second embodiment, the coder codes mov- 
ing-picture data by referencing a successfully-received 
frame without waiting for the next I frame, as in the first 

25 embodiment, ensuring good coding efficiency. In addi- 
tion, the ability to adaptively switching the reference 
frame update mode according to the transmission sta- 
tus gives optimal coding efficiency. 

While there is no transmission error, the reference 

30 frame picture is updated each time a frame is received. 
Therefore, coding efficiency is not decreased even 
when the round trip delay is large. When a transmission 
error occurs less frequently, the Negative mode is used 
to prevent picture quality from being degraded. When a 

35 transmission error occurs frequently, the Active mode, 
which stables picture quality with no sudden degrada- 
tion, is used to maintain picture quality. 

In either update mode, only the ACK signal is used 
for the decoding status signal. This decreases the com- 

40 plexrty involved in mode switching. In addition, the ability 
to send a correct frame number or a correct block 
number eliminates the need to wait for correct data after 
an error, minimizing delays. Because the sending 
device uses the decoding status signal to check the 

45 transmission status in order to determine whether the 
update module must be changed, no additional informa- 
tion is needed to check the transmission status. 

Operation of the receiving device does not depend 
on the update mode of the reference frame. Mode 

so switching is performed only on the sending device, and 
information on mode switching need not be sent to the 
receiving device. This does not require the re-configura- 
tion of the receiving device but prevents additional 
transmission load. It should be noted that mode switch- 

55 ing is performed as a result of a change in the transmis- 
sion line status and that information exchange between 
the sending device and the receiving device requires 
error correction processing. It includes error correction 
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codes, error protection, or retransmission. Thus, elimi- 
nation of the need to send switching information results 
in an increase in system operation efficiency. 
Variations of the second embodiment are: 

(1) Although, in the second embodiment, the trans- 
mission status is one of two stages, either "good" or 
"bad," and therefore there are two update modes. 
There may be three or more transmission status 
modes. For example, a new stage "extremely bad" 
may be added. If there is an error in almost all con- 
secutive pieces of data, coding may be done at a 
lower level to reduce the amount of coding and to 
reduce the probability of errors. 

(2) Although only the sending device performs 
mode switching and no mode-switching information 
is sent to the receiving device in the second embod- 
iment, the present invention is not limited to this 
configuration. Mode-switching information may be 
sent to the receiving device to allow it, for example, 
to clear the frame memory module. For-example,Jn 
the Active mode, data that is earlier than the frame 
(or block) selected for the reference frame may be 
deleted. 

(3) In the second embodiment, mode switching may 
be done by the receiving device. The acknowledg- 
ment signal is used to detect an error ratio only on 
a frame basis or on a block basis, while the receiv- 
ing device may detect an error ratio on the trans- 
mission line, more flexibly, on a bit basis with the 
use of the CRC. The receiving device may specify a 
mode based on a detected error ratio, or may send 
a detected error ratio to the sending device to allow 
it to perform mode switching based on the received 
error ratio. 

(4) In the second embodiment the transmission 
status is determined to be bad if an error is 
detected in L or more consecutive frames or in L or 
more consecutive blocks. It is also possible to 
determine that the transmission status is bad when 
a decoding error occurs consecutively in the same 
block of a plurality of frames continuously. 

(5) In the second embodiment, the transmission 
status is determined by the number of decoding 
errors detected through the decoding status signal, 
and the mode is switched automatically according 
to the determined status. The transmission status 
may also be determined manually. For example, a 
user on the sending device may switch the update 
mode or may send a switching signal to allow a user 
on the receiving device to switch the mode. 

(C) Other embodiments 

(1) The moving-picture coder in each of the above 
embodiments may be implemented as a hardware 
unit or a software program. 

(2) In each of the above embodiments, transmis- 



sion data is moving-picture data. Transmission data 
may be voice data or binary data. 

As described above, according to the present 

5 invention, the picture decoder sends the decoding sta- 
tus signal and a frame number (or frame number and 
block number) to the picture coder. The picture coder 
controls the update of a reference picture based on the 
decoding status signal and the frame number (or frame 

10 number and block number). If a transmission line over 
which coded data is transmitted is low in error frequency 
and high in transmission quality, high picture quality is 
ensured. Conversely, if the line is high in error frequency 
and low in transmission quality, P frames may be 

75 decoded before the next I frame arrives. 

While there has been described what are at present 
considered to be preferred embodiments of the inven- 
tion, it will be understood that various modifications may 
be made thereto, and it is intended that the appended 

20 claims cover all such modifications as fall within the true 
spirit and scope of the invention. 

Claims 

25 1. A picture coder which includes coding means for 
coding input pictures through predictive coding, one 
at a time, based on a reference picture and for out- 
putting coded data; transmission means for trans- 
mitting the coded data coded by the coding means 
so and a frame number thereof to a picture decoder; 
and reference picture updating means for control- 
ling updating of the reference picture based on a 
decoding status signal sent from the picture 
decoder and an associated frame number, said pic- 
as ture coder comprising: 

decoding status determination means which 
uses an acknowledgment signal as the decod- 
ing status signal and which determines that a 
40 decoding error has occurred if the acknowl- 

edge signal is not received within a specified 
time. 

wherein, as a next reference picture, the refer- 
ence picture updating means selects one of 
45 pictures transmitted correctly before the decod- 

ing error occurred. 

2. A picture coder as set forth in claim 1 . wherein the 
reference picture updating means divides each 
so frame of the input picture into a plurality of blocks 
and updates the reference picture using the frame 
number of the frame and a block number of each of 
subdivided blocks. 

55 3. A picture coder as set forth in claim 1 . wherein the 
reference picture updating means records a 
number of the frame for which the acknowledgment 
signal is received as a last-received number and, 
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upon detecting the decoding error, sets up the 
frame or the block with the last-received number as 
the reference picture. 

4. A picture coder as set forth in claim 2, wherein the s 
reference picture updating means records a 
number of the frame for which the acknowledgment 
signal is received as a last-received number and, 
upon detecting the decoding error, sets up the 
frame or the block with the last-received number as 10 
the reference picture. 

5. A picture coder as set forth in claim 1 , wherein the 
reference picture updating means comprises: 

is 

a frame number storage module for storing the 
frame number of the input picture, coded by the 
coding means, as a candidate of the reference 
picture; and 

a deleting module for deleting the frame 20 
number on which the decoding error occurred 
and frame numbers of input pictures coded 
after the decoding error, 
wherein, the picture with the frame number 
stored in the frame number storage module is 25 
selected as the reference frame. 



reference picture updating means comprises: 

a frame number storage module for storing the 
frame number of the input picture, coded by the 
coding means, as a candidate of the reference 
picture; and 

a deleting module for deleting the frame 
number on which the decoding error occurred 
and frame numbers of input pictures coded 
after the decoding error, 
wherein, the picture with the frame number 
stored in the frame number storage module is 
selected as the reference frame. 

9. A picture coder as set forth in daim 1 , further com- 
prising: 

status determination means for determining a 
transmission line status based on the decoding 
status signal and the frame number; and 
update mode switching means for switching 
and controlling, in response to a determination 
result of the transmission line status determi- 
nation means, an update method of the refer- 
ence image updated by the reference picture 
updating means. 



6. A picture coder as set forth in claim 2, wherein the 
reference picture updating means comprises: 

30 

a frame number storage module for storing the 
frame number of the input picture, coded by the 
coding means, as a candidate of the reference 
picture; and 

a deleting module for deleting the frame 35 
number on which the decoding error occurred 
and frame numbers of input pictures coded 
after the decoding error, 
wherein, the picture with the frame number 
stored in the frame number storage module is ao 
selected as the reference frame. 

7. A picture coder as set forth in claim 3. wherein the 
reference picture updating means comprises: 

AS 

a frame number storage module for storing the 
frame number of the input picture, coded by the 
coding means, as a candidate of the reference 
picture; and 

a deleting module for deleting the frame so 
number on which the decoding error occurred 
and frame numbers of input pictures coded 
after the decoding error, 
wherein, the picture with the frame number 
stored in the frame number storage module is ss 
selected as the reference frame. 

8. A picture coder as set forth in claim 4, wherein the 



10. A picture coder as set forth in claim 2, further com- 
prising: 

status determination means for determining a 
transmission line status based on the decoding 
status signal and the frame number; and 
update mode switching means for switching 
and controlling, in response to a determination 
result of the transmission line status determi- 
nation means, an update method of the refer- 
ence image updated by the reference picture 
updating means. 

11. A picture coder as set forth in claim 9, wherein, 
when the transmission tine status determination 
means determines that the transmission line status 
is good, the update method switching means 
switches the update method from the update 
method by which the reference picture is updated 
when there is no decoding error to the update 
method by which the reference picture is changed 
to one of previous pictures when there is the decod- 
ing error. 

12. A picture coder as set forth in claim 10, wherein, 
when the transmission line status determination 
means determines that the transmission line status 
is good, the update method switching means 
switches the update method from the update 
method by which the reference picture is updated 
when there is no decoding error to the update 
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method by which the reference picture is changed 
to one of previous pictures when there is the decod- 
ing error. 

13. A picture coder as set forth in claim 9. wherein, s 
when the transmission line status determination 
means determines that the transmission line status 

is bad, the update method switching means 
switches the update method from the update 
method by which the reference picture is changed 10 
to one of previous pictures when there is the decod- 
ing error to the update method by which the refer- 
ence picture is updated when there is no decoding 
error. 

15 

14. A picture coder as set forth in claim 10, wherein, 
when the transmission line status determination 
means determines that the transmission line status 
is bad, the update method switching means 
switches the update method from the update 20 
method by which the reference picture is changed 

to one of previous pictures when there is the decod- 
ing error to the update method by which the refer- 
ence picture is updated when there is no decoding 
error. 2 s 

1 5. A method of transmitting coded picture data com- 
prising 

coding input pictures sequentially by predictive 30 
coding, based on a reference picture, so as to 
produce coded data for the pictures succes- 
sively; 

transmitting the coded data to a picture 
decoder; and 35 
controlling updating of the reference picture 
based on a decoding status signal sent from 
the picture decoder and an associated frame 
number, characterised by 

using as the decoding status signal, acknowl- <o 
edgment signals (ACK) from the picture 
decoder which respectively indicate successful 
decoding of the data for an individual one of the 
pictures by the decoder, thereby indicating that 
a decoding error has occurred if not received as 
within a specified time, and 
selecting as the reference picture, one of pic- 
tures transmitted correctly before the decoding 
error occurred. 

SO 

16. A picture coder operative in accordance with the 
method claimed in claim 15. 

17. A picture decoder operative to decode coded pic- 
ture data produced in accordance with the method 55 
claimed in claim 15, and to produce said acknowl- 
edgement signals in response to successful decod- 
ing of the coded data for the pictures respectively. 
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